Best rank one approximation of real symmetric tensors 
can be chosen symmetric 



S. Friedland 

Department of Mathematics, Statistics and Computer Science 
University of Illinois at Chicago 
^ '■ Chicago, Illinois 60607-7045, USA 

e-mail:f riedlan@uic . edu 

O 

O ; October 25, 2011 

(N 

Abstract 

I We show that a best rank one approximation to a real symmetric tensor, 

pLn ' which in principle can be nonsymmetric, can be chosen symmetric. Further- 

(-H I more, a symmetric best rank one approximation to a symmetric tensor is unique 

■ if the tensor does not lie on a certain real algebraic variety. 

~' 2010 Mathematics Subject Classification. 15A18, 15A69, 46B28, 65D15, 

65H10, 65K10 

Key words. Symmetric tensor, rank one approximation of tensors, uniqueness of 
! rank one approximation. 

00 

^ '. 1 Introduction 

^ Denote by l^"ix---x"d — ^'^^-J^'^i the tensor products of W^'^, . . . , M"<* for an integer 

d >2. T = G M.nix...xna called a d-tensor. Let [d] := {l,...,d}. For 

Xj = {xij, . . . , Xjijj)'^ G K"-' , j G [d\, the decomposable tensor (8>j=iXj = xi (X" . . . x^ 
is given as [11^=1 ^ ]^"ix---x"d_ ^ decomposable tensor is a rank one tensor if 
^ ' and only if Xj 7^ for each j G [d]. 

c5 ■ On M«ix...xnd jggj^g i^j^g^ product (5, T) := Ei,G[n,] jeM •5ii,...,id*n,...,id and 

the Hilbert-Schmidt norm ||T|| := \/{T,T). For x G M" let ||x|| := Vx^x be the 
I2 norm. Denote by S"^"*^ := {x G M", ||x|| = 1} the n — 1 dimensional unit sphere. 
Then a best rank one approximation of T is a decomposable tensor solving the 
minimal problem 

min ||T-s®j=iXj|| = ||T-a(g)''=iUj||. (1.1) 

It is well known 



max 



For matrices, i.e. d = 2, a best rank one approximation of A G R^-xn jg 
given by (Ji(74)xy^, xx^ = yy^ = 1, where (Ti{A) is the maximal singular value 
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of A, and Ay = ai{A):x., A~^ :k = ai{A)'y. A best rank one approximation is 
unique if and only if cri{A) > a2{A). Assume that A G M"^"- is symmetric. Let 
•^i(^) > • • • > ^n{A) be the n-eigenvalues of A, counted with their multiphcities. 
Then cri{A) = max(|Ai(j4), |A2(^)|). Furthermore, there always exists a symmetric 
best rank approximation. Moreover, there exists a nonsymmetric best rank one 
approximation if and only if A„(^) = —Xi{A) < 0. 

Assume now that T is a d-mode tensor with d > 3. Let a be a subset of [d] 
of cardinality 2 at least. We say that T = [tii,...,^^] G i^"ix---"d symmetric with 
respect to a if = Ug for each pair {p,q} C a and the value of ti^^^^^^i^ does 
not change if we interchange any two indices ip, iq for p,q G a and for any choice 
ij G [n'j],j G [d]. We agree that any tensor T is symmetric with respect to each 
{i}, for i G [d]. T is called symmetric if a = [d]. Denote by Sym(n, d) all d-mode 
symmetric tensors in Let T G ]^"ix---x"d given. Clearly, there exists a 

unique decomposition of [d] to a disjoint union of nonempty sets Uj^iCtj such that 
the following conditions hold. 

• For each j G [m] the tensor T is symmetric with respect to aj. 

• For 1 < j < k < m and two indices p G aj,q G the tensor T is not 
symmetric with respect to {p,q}- 

We call [d] = U^^aj the symmetric decomposition for T- 
The main result of this paper is. 

Theorem 1 Let T G \ {0} be given. Assume that [d] = U^^Oj is the 

symmetric decomposition for T- Then there exist a best rank one approximation A 
to T such that A is symmetric with to each aj . 

It is not difficult to show that to prove the above theorem it is enough to show 
the above theorem for symmetric tensors. Furthermore, it is enough to show the 
above theorem for T G Sym(2, d). Finally we show that there exists an real algebraic 
variety Si(n, d) C Sym(n, d) such that for T G Sym(n, d) \ 5]i(n, d) a best rank one 
symmetric approximation is unique. 

We now describe briefly the contents of our paper. In §2 we summarize the well 
known results of best rank one approximation for real matrices that are used in this 
paper. In §3 we discuss certain basic results on best rank one approximation of real 
tensors. In §4 we give a complete characterization of tensors T G Sym(2, 3) which 
have nonsymmetric best rank one approximation. In §5 we prove Theorem [1] by 
first showing the case where T is a symmetric tensor, (Theorem [9]) . In §6 we show 
that a "generic" symmetric tensor has a unique best rank one approximation. The 
last section uses some facts from algebraic geometry, and probably the most difficult 
section of this paper. 

2 Best rank one approximation of matrices 

We recall briefly the needed results on best rank one approximation of matrices 
A = [ajj] G M™^" [6]. Assume that r = ranked. Then A has exactly r positive 
singular values o"i(A) > . . . > c^r^A) > 0. ai{A)'^ > ■ ■ ■ > are the positive 

eigenvalues of either AA"^ or A'^ A. 

(Ji{A) = max x'^Ay. (2.1) 

xGS™-i,yeS"-i 
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The left and the right singular pair of singular vectors u G S™" , v E S" corre- 
sponding to o-i(^) are given by equalities 

A^u = (Ti(^)v, = ai{A)u, u G S™-\ v € 8""^ (2.2) 

Hence cri{A) = A\r. Furthermore any best rank one approximation of A in the 
Frobenius norm ||j4|| = y/ tT{A'^ A) is of the form al{A)vJ^r for some pair of singular 
vectors u, v corresponding to ai{A): 

min ||A - sx'^yll = P - cri(^)u'^v||. (2.3) 

Recall that Sym(n, 2) C R"^*^ is the space of symmetric matrices. The following 
result is well known and we bring its proof for completeness. 

Proposition 2 Let A £ Sym(n,2). Then cri{A) = max(|Ai(A)|, |A„(A)|) and 
A has a symmetric best rank one approximation. Suppose furthermore that ^4 / 0. 
Then 

1. Any best rank one approximation to A is symmetric if and only if Xi{A) / 

-Xn{A). 

2. Assume that Xi{A) = —Xn{A). Then ai{A) = Xi{A). Furthermore, ai{A)vJv, 
where (j2.2p holds, is a nonsymmetric best rank one approximation of A if nei- 
ther u norw are eigenvectors of A anc?u,v are eigenvectors of A^ correspond- 
ing to Xi{A)'^. 

Proof. Since A~^ A = A"^ the singular values of A are |Ai(j4)|,i E [n]. Hence 
cri{A) = max(|Ai(^)|, |A„(A)|). We now show that there exists a symmetric best 
rank one approximation of A. Clearly, it is enough to consider A ^ 0. Assume that 
(7i{A) = \Xj{A)\,j S Let v G S""^ be an eigenvector of A corresponding to 

Xj{A). Hence u = ^x^^^v and ai{A)\i^v = Xj{A)\r^v, which is a symmetric best 
rank one approximation of A. 

Assume now that A ^ 0. Note that the assumption that |Ai(j4)| ^ |A„(j4)| is 
equivalent to Xi{A) ^ — A„(A). Assume first that Xi{A) ^ — A„(A). Then there 
exists a unique j G {l,?^} such that cri{A) = |Aj(^)|. As a right singular vector 
V G S*^"^ of A is an eigenvector of corresponding to Xj{A)'^ it follows that v is an 
eigenvector A. Hence the above argument show that ai{A)u^\r is symmetric and[T] 
holds. 

Assume now that Xi{A) = — A„(^). If v is an eigenvector of A corresponding to 
Xj(A),j G then u is also eigenvector of A corresponding to Xj{A), and vice 

versa. In this case ai{A)u^\r is a symmetric best rank one approximation. Assume 
that V is not an eigenvector of A. Since v is a right singular vector corresponding 
to cri{A) it follows that v is an eigenvector of corresponding to the eigenvalue 
Xj{A)'^. Similar claim holds for u. Hence {A\r)~^\r is a nonsymmetric best rank 
approximation and [2] holds. □ 

Lemma 3 

1. A £ l^»"X" fias a unique rank one approximation for m,n > 2, unless A lies 
on a subvariety of codimension two. 
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2. A €z Sym(n, 2) has a unique rank one approximation for n > 2, which is 
symmetric, unless A lies on a subvariety of codimension one in Sym(n, 2). 

Proof. To prove the first part of the Lemma we use the singular value decom- 
position. Assume without loss of generality that 2 < m < n. Then each matrix 
A e M'"^" is of the form UDV~^ where U = [ui,...Um] G R^x'" is orthogonal, 

V = [vi, . . . , Vm] G i^"x™ has m-orthogonal columns and D = diag(di, . . . , dm),, 
where di > . . . > dm > 0. So cri{A) = di,i G [m], and the columns i oiV and U are 
right and the left singular vectors respectively corresponding to the singular value di. 
Note that if di > . . . > dm > then each column of V is determined up to ±, and af- 
ter V is fixed, then U is determined uniquely. In this case cri{A) = di > a2{A) = d2 
and A has a unique rank one approximation. A has no unique approximation if and 
only if di = d2. The generic case for this situation is 

di = d2 > dz > ■ ■ ■ > dm > ^- (2.4) 

The equality di = d2 means that we loose one parameter. The columns 3, . . . , m of 

V are determined uniquely to ±1. The first two columns of V are not determined 
uniquely. What is determined uniquely is the two dimensional subspace V C M" 
which is orthogonal the the columns 3, . . . , m of F and the null space of A. We can 
choose as a first column vi of V any unit vector in V. The the second column V2 
of y is a unit vector in V which is orthogonal to vi. So V2 is determined uniquely 
up to a sign. Recall that Uj = d~^^Vj,i G [n]. Hence the set A = UDV^ of the 
above form, where the entries of D satisfy ()2.4p . is a manifold <I>(m,n) C M^xn. q£ 
codimension two in W^^"". It is left to show that there is a nonzero polynomial Q 
in the entries of A which satisfies the following conditions. First, Q vanishes on 
$(m, n). Second, for each A £ $(m, n) the exists a neighborhood O C M'^^n q£ ^ 
such that the zero set of Q on O is equal to O n <I>(m,n). Consider the symmetric 
matrix B = AA~^ . Let Dis(i?) be the discriminant of the characteristic polynomial 
of B. Then Dis(i3) vanishes if and only if B has a multiple eigenvalue. In particular 
Dis(i?) vanishes on <I>(m, n). Fix A G ^{m,n). Assume that C G M^^n -g ygj.y 
close to A. Then a2{C) > ... > ct™(C) > 0. So Q{CC^) = if and only if 
<Ti(C) = (72 (C). This establishes the first part of the Lemma. 

Recall that the set of all A G Sym(n, 2) having at least one multiple eigenvalue 
is a variety A„ of codimension two, e.g. [5]. (This follows from the arguments 
of the first part of the Lemma. A„ is the zero set of the discriminant of the 
characteristic polynomial of A.) Assume that A G Sym(n,2) \ A„. (So A ^ 0.) 
Proposition [2] yields that A has a unique symmetric rank one approximation if and 
only if Ai(^) / —Xn{A). It is left to show that all matrices A G Sym(n, 2) \ A„ 
satisfying Xi{A) + A„(y4) = lie on a variety of codimension one. This follows 
from the spectral decomposition A = UDU^, where U ia an orthogonal matrix 
and D = diag(di, . . . ,dn),di > ... > dn- Note that the columns of are deter- 
mined uniquely up to a sign. Hence the set of all A G Sym(n, 2) \ A„ satisfying 
\i{A) + XniA) = is a manifold of codimension one. It is left to show that that 
this manifold is a zero set of some polynomial in A. Recall that for A G Sym(n, 2) 
the matrix A In + In ^ A G Sym(n^,2), where In G Sym(n,2) is the identity 
matrix and A0 B is the Kronecker tensor product, have eigenvalues \i{A) + Xj{A) 
for i,j G [n]. Hence the zero set of det{A0 In + In'i^ A) includes the above manifold. 
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This concludes the proof of the second part of the Lemma. □ 



3 Preliminary results on best rank one approximation 
of tensors 

Recall that oo-Schatten norm of T = ^ with respect to the £2 

norm on each factor is given by 

||r||oo,2:= max {{T x ^ti^i)!- 

x,eS"i-i,ie[d] 

Since -S"-i = S"-i it follows that 

ni,...,na 

ll'7'l|oo,2 := max ti^,...,iaXii,i ■ ■ ■ Xi^^d- (3.1) 

x.=(xi,„...,x„^,,)TeS".-i,ie[d] . 5^. 

Zl — ... — 

See for example p] for a modern exposition on tensor norms and [3] for simple 
geometrical properties of cross norms. Note that for matrices, i.e. d = 2, \\A\\^^2 
is the operator norm ||A||2 = ai{A), where A € M™^" viewed as a linear operator 
Ay from M" to M". 

Let /3 C [d] be a nonempty set and assume that Xj = {xij, . . . , Xnjj)^ G M"^ for 
j E /3. Denote by T x (^^j^i^Xj the contracted d — |/3| tensor 

^jel3^j-= Yl ^il,■■■,ici'[lx^^J ^^ke[d]\i3^'"'- (3.2) 

Note that if /? = [d] then T x (XijgfrfjXj = (T, (8'j(=[(i]Xj). Let /3 be a nonempty strict 
subset of [d]. By considering the maximum in (|3.2p as a maximum on Xj first on 
j £ [d]\ (3 and then on j £ /3 we deduce 

||7'||oo,2 = max ||T X (g)j-g^Xj||oo,2- (3.3) 

Suppose that f3 = [d]\ {p, q}, where 1 < p < q < d. We view T x as a matrix 

in W^p'^^i. Hence 

||T||oo,2 = max o-i(T X Ojg[rf]\{p^g|Xj). (3.4) 

XjGS"i ,ie[d]\{p,g} 

The following result is well known and we bring its proof for completeness. 

Lemma 4 Let T be a given tensor in Then a 0f=i Uj, where 

Uj E S"*^"*^, i E [d], js a best rank one approximation ofT if and only if the following 
conditions hold. First a = ib||7'||oo,2- Second the function {T,0j,^[d]Xj) attains its 
maximum or minimum on S"^^""^ x . . . x S"''^"'^ at (ui, . . . , u^). In particular 

T X %g[d]\{i}Uj = Xui, i E [d], (3.5) 

where A = ±||T||2,oo O'lT-d Uj E S"'^^ for i E [d]. Suppose furthermore that T 
is symmetric with respect to 1 < p < q < d. Then there exist a best rank one 
approximation which is symmetric with respect with respect to {p,q}. 
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Proof. Let Xj S S"'~^ for i G [d]. Then (^jgj^^jXj is an orthonormal vector 
in 'S'i^i^^'- Let U := span((8)jg[(;]Xj) and U"*- be the orthonormal complement 
of U in (^f^^W^\ The orthogonal projection of 7~ on U is given by Pu{T) = 
(T, (8)jg[rf]Xi)(g)jg[rf]Xi, and llPuCT")!! = \{T,'S>i(z[d]Xi)\. It is well known that min^gK ||T- 
s ^f^i Xj|| = ll-Pu-L (T)||. The Pythagoras theorem yields that 

lirii = ||Pu(r)f + ||Pu4r)f = (r,0.eHX.>' + \\PuAT)f. (3.6) 

Hence a minimal solution of the left-hand side of (ll.ip gives rise to a maximum or 
minimum of (T, on S"^"-^ x . . . x S"'*"^. 

Consider the maximum problem (ll.2p . Use Lagrange multipliers for the function 
(T, (8'je[rf]Xj) — X^jgfrf] -^jxjxj to deduce that a maximum solution satisfies 

T X ^je[d]\{i}Uj = AjUi, i G [d\. 

Hence {T,(>^j£[d]^j) = AjU^Uj = Aj for each i G [d]. Therefore a best rank one 
approximation a(5Djg[(i]Uj satisfies a = ||T||oo,2 and ()3.5p . where A = a and uj G S"j~-^ 
for j G [c?]. Similar results hold for the minimum of {T,(^je[d]^j) on S^i""*^ x . . . x 

Assume now that T is symmetric with respect to two indices p < q. So Hp = Uq. 
Assume that a^f^^ Uj is best rank one approximation. Let Uj = span(uj),i G [d\. 
Note that A := T x '^i£[d]\{p,q}Ui is a symmetric matrix. As best rank one approxi- 
mation of a j4 can be chosen symmetric we deduce that we can choose Up, Ug G S"*""^ 
such that Uq G {up, — Uj,}. Hence there exist a best rank approximation of T which 
is symmetric with respect to {p,q}- □ 



Corollary 5 Let T 7^ be a given tensor in ]^"ix---x"-d_ Then rank one tensor 
A G ]R"ix - xnd ^g_j^ rank one approximation ofT if and only if 

{T,A) = \\T\\l,2 = \\Af. (3.7) 
The following lemma is straightforward. 

Lemma 6 Let T G iR"-ix - xnfj assume that a<Sij£[d] Uj is a best rank one 
approximation of T. Suppose that T is symmetric with respect to a C [d] . Let 
o" : [fi] — )• [d\ be a permutation which is identity on [d\ \ a. Then a(8>jg[d] x^-q) is a 
best is rank one approximation ofT. In particular, if a®j^^d] Uj is unique best rank 
one approximation ofT then a(S>j<=[d] Uj is symmetric with respect to a C [d]. 

Lemma [3] suggests the following conjecture. 

Conjecture 7 Let d > 3,nj >2,j G [d] be integers. Then 

1. T G ^nix...xna /j^g ^ n^iiqug rank one approximation, unless T lies on a 
subvariety. 

2. T £ Sym(n, d) has a unique rank one approximation, which is symmetric, 
unless T lies on a subvariety. 
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We remark that T. Kolda in her lecture [7j stated a stronger version of the second 
part of Conjecture [71 Namely:" rank-r symmetric /factorization/ of a symmetric 
tensor is unique even without the symmetry condition, under very mild conditions" , 
although she did not specify the nature of the mild conditions. 

Proposition 8 Assume that the second part of Conjecture [H holds for some 
integers n > 2,d > 3. Then each T G Sym(n, d) has a best rank one symmetric 
approximation. 

Proof. Assume that ^„ C Sym(n, d) is the variety of all symmetric tensors which 
do not have a unique best rank one approximation. So each T G Sym(n, d) \ has 
a unique best rank one approximation A{T) which is symmetric. Assume now that 
T G ^n- As is a variety, there exists a sequence Tk,k £ N which converges to T, 
and Tk^'^n for all /c G N. Use Corollary [5] to deduce 

lirilL.a = lim \ml^ = hm {%,A{Tk)) = lim \\A{Tk)f . 

fe— >oo k-^oo fe— >oo 

So A(Tk) is a bounded sequence in Sym{n,d). Hence there exists a subsequence 
A{Tki) which converges to a rank one symmetric tensor A which satisfies IIT"!!^ 2 ~ 
(T, A) = ll^lP- Hence by CorollaryO^ is a symmetric best rank one approximation 

of r. □ 

A weaker version of the second part of Conjecture [7] is: 

Theorem 9 Every symmetric tensor T G Sym(n, d) has a symmetric best rank 
one approximation for integers n > 2, d > 3. 

Note that the above theorem is a special case of Theorem [TJ We will first prove 
Theorem [H and using it we will prove Theorem [TJ 

Lemma 10 Let k > 2. Assume that Theorem [5| holds for n = 2 and for all 
positive integers d in the interval [2, k]. Then Theorem\E holds for all integers n > 3 
and d G [2, k]. 

Proof. We prove our Lemma by induction on k. In view of Proposition [2] The- 
orem [9| trivially holds for k = 2. Assume that > 3 and suppose that we proved 
the Lemma for A; = — 1. Assume that Theorem [9] holds for Sym(2,A^). Let 
T = [tii,...,ijv] G Sym(n,A^) and n > 3. Suppose that ||T||2,oo = \{T,<^j^[d]Vj)\, 
where v^- G S^-^ for j G [A^]. Let 5 := T x v^v G Sym(n, A^ - 1). So ||5||2,oo = 
||7'||2,co- Our induction assumption implies that there exists u G S""^ such that 
115112 00 = |(5,(8)j£[^_i]Uj)|, where uj = u for j £ [N — 1]. Let u^v = v^y. Then 
<^jG[Af]Uj)|. If UN — ±u it follows that that (T, 0jg[Ar]Uj) '^je[N] Uj 
a best rank one symmetric approximation of T, and we are done. Suppose that 
u^r 7^ ibu. So span(u, ujv) is two dimensional. By changing an orthonormal basis 
in R" we may assume without loss of generality that span(u, u^r) = span(ei,e2), 
where e^ is the j-th column of the identity matrix Let T' = [^ii,...,iiv]n,...,iive[2] £ 
Sym(2,A^). So ||7'||oo,2 = ||T''||oo,2- Our assumption implies that there exists 
w' G S-*- such that ||T'||oo,2 = \{T' ,®j^[N]^j)\, where w^- = w' for j G [A^]. Let 
MVj = w' ® 0Ar_2 G S^^"^. Then (T, ^je[N]'^j) ^je[N] is a symmetric best rank 
one approximation of T. □ 
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4 Best rank one approximations of T G Sym(2,3) 



Theorem 11 Let T = [tij^k] £ Sym(2,3). Then each best rank one approxima- 
tion of T is symmetric, unless T is a nonzero tensor proportional to the following 
one. 

ti,i,i = cos0,ti,i,2 = sm6',ti_2,2 = - COS 6*, ^2, 2,2 = -sinO, Be [0,2it). (4.1) 

For the above tensor there is a best rank approximation of the form u (g) v (S" w(u, v) 
where u,v G are arbitrary and w(u, v) G is uniquely determined by u,v. 
Furthermore, the above tensor has three symmetric best rank one approximations. 

Proof. Since T = has a unique best rank one approximation - T, we assume 
that T 7^ 0. Suppose that T has a best rank approximation cx y (g) z, x, y , z G S^ 
which is not symmetric. (Note that c 7^ 0.) By permuting x, y,z we can assume 
that y 7^ ibz. Let ^(x) := 7~ x x G Sym(2,2). (Because T is symmetric, it is not 
important which index we contract.) Hence cyz^ is best rank one approximation 
of the symmetric matrix ^(x). (Note that ^(x) 7^ 0.) Proposition [2] yields that 
Ai(^(x)) + A2(^(x)) = 0, which is equivalent to tr(A(x)) = 0. Observe next that 
^(x)^ is a scalar matrix, i.e. ^(x)^ = Ai(^(x))^/2- Let u G S^ be arbitrary 
and V := ||^(x)u|| ^(x)u(g S^). Proposition [2] yields that || 74(x)u||uv~'^ is a best 
rank approximation of ^(x). Choose u so that u, v {x, — x}. Then c'xv^ is a 
nonsymmetric best rank one approximation of ^(u). The previous arguments show 
that tr(^(u)) = 0. Since x and u are linearly independent, it follows that the two 
frontal section := [tij^k]i=j=i ^ Sym(2, 2) have trace zero. Taking in account the 
T G Sym(2, 2) we deduce that T is proportional to the tensor given by ()4.ip . 

Assume that T is of the form (j4.ip . Let 



cos 6 sin 9 
sin 6 — cos t 



sin 6 — cos I 
- cos 6 — sin ( 



be the two frontal sections of T. Then 

A((cos0,sin(/.)^) = cos 0^1(0) + sin (/>^2(^) = Ai(0 -(/.). (4.2) 

So Ai(yl(u)) = — A2(^(u)) = 1 for every u G S^. Hence any best rank one approxi- 
mation of A{u) is of the form vw(u, v)^, where v G S^ and w(u, v) := A{u)v. This 
shows that any best rank approximation of T is u (g) v (g) w(u, v) as claimed. 

It is left to show that T has exactly 3 different best rank one symmetric ap- 
proximations. In view of ()4.2p . by changing an orthonormal basis in we may 
assume that 9 = 0. The condition that T has a symmetric best rank approxima- 
tion means that we need to choose x G S^ such that A(x) = itx. Note that if 
A(x)x = X then ^(— x)(— x) = — (— x). Hence we need to find all x G S^ such that 
A(x)x = X = (cos 4>, sin <j)). This condition gives rise to the following three solutions 

□ 
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5 Proofs of Theorems [9] and [T] 



Lemma 12 Let d >2 be an integer and T = [iji,...,^^] G Sym(2, (i)\{0}. Assume 
that T has a nonsymmetric best rank one approximation. Then for each i^, . . . E 
[2] the symmetric matrix [ti,j,i3,...,ijj]^=j=i G Sym(2, 2) has zero trace. 

Proof. For d = 2 the lemma follows from Proposition [2l For d = 3 the lemma 
follows from Theorem [TTJ We prove the lemma by induction on d > 3. Suppose 
that the lemma holds for d = N > 3. Assume that d = N + 1. Suppose that 
A = aiXij^iXj, Xj G S^, j G [d], a 7^ is a nonsymmetric best rank one approximation 
of T. Since T is symmetric it follows that for each permutation cr : [d] — t- [d] 
the decomposable tensor o <8)j=i ^a{j) is best rank one approximation of T which 
is nonsymmetric. Hence, without a loss of generality we may assume that x^^ ^ 
±Xd-i. Fix the vectors xi, . . . ,Xd_3 and consider ^(xi, . . . ,Xrf_3) := Tx £ 
Sym(2, 3). Clearly, is a nonsymmetric best rank one approximation 

to T(xi, . . . ,Xd_3). 

Theorem 1111 yields that a best rank one approximation of ^(xi, . . . jX^s) can 
be chosen of the form a(8)w(u,v)(S'u(g)v where u, v are arbitrary vectors in S"*^ and 
w(u, v) G S^ Fix the vector v = {vi,V2y G S^ Then A := a(((g)^^^Xj) (g) w(u, v) (g) 
u) is best rank one approximations of 7~(v) := T x v. Observe that A is not sym- 
metric for u 7^ ibxi. Hence the induction hypothesis yield that the tensor 7~(v) = 
+ tii,...,id-i,2V2] S Sym(2,(i — 1) satisfies that the assumption of the 
lemma. I.e., for any is,... ,id-i G [2] the matrix [tij.ja,...,^^.!,!^! +tji,...,id_i,2^'2]jLj=i 
has zero trace. Let v = (1, 0)^, (0, 1)^ to deduce that the lemma holds for T. □ 

Proof of TheoremO We first prove the theorem for T = [tii,...,id] G Sym(2, d)\ 
{0}. Suppose first that for some is,...,id G [2] the 2x2 symmetric matrix 
[^j,j,i3,...,id]jje[2] does not have trace 0. Then Lemma [12] yields that each best rank 
one approximation tensor of T is symmetric. 

Assume now that for each i^, . . . ,id £ [2] the 2x2 symmetric matrix [ti,j,iz,...,ici\i,j£[2] 
has trace 0. Let S = [si^^...^i^ G Sym(2,(i) be the following tensor. si^...^i = 1 and 
all other entries of S are zero. Then for any e 7^ the tensor T + eS is symmetric, 
and the trace of the matrix + esj,j,i,...,i]jjg[2] is e. Lemma [T2] yields that 

T + eS has best rank one approximation A{£) G Sym(2,(i). Recall that 

||^(e)f = \\T + eS\\l^ = {T + eS,A{e)). (5.1) 

(fOj) yields that 

ll^(e)ll<lir + 6^ll<lir|| + kii|5|| = ||r|| + |e|. 

Consider the bounded sequence Am := A{-^) G Sym(2,d),m G N. There exists 
a subsequence Am^,i G N, such that Ymii^ooAmi = A £ Sym(2,d), where ^ is a 
decomposable tensor. (|5.ip yields that ^ is a best rank one approximation of 7~. 
This completes the proof of the theorem for T G Sym(2, d). 

Assume that n > 2. Use Lemma [TOl to conclude that each T G Sym(n,(i) \ {0} 
has a symmetric best rank one approximation. □ 

Proof of Theorem [H Let T G ]R"ix-x"d \ {0}. Assume that [d] = U^^^aj 
is the symmetric decomposition for T. We prove the theorem by induction on k. 
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Assume that k = 1, i.e. T S Sym(n, d). Then the theorem follows from Theorem [9j 
Suppose that the theorem holds for k € [m], where m > 1 and assume that k = m+1. 
Permute the factors in ^j^-^W^^ to assume that qi = {1, ...,/},/< d. Suppose that 
a ®j=i ^jj^j S S"j~^,j G [d\ is a best rank one approximation of T. Recall that 
a Xj is a best rank one approximation of T(xi, . . . , x^) := T x (E>j=x^j- Fur- 

thermore p.3p yields that ||T||2,oo = . . . ,x/)||2,oo- Observe that T(xi, . . . ,x/) 

is symmetric with respect to 02, . . . , a^. Hence the induction hypothesis yields that 
there exists a best rank one approximation b <Sij^i_^_i yj of T(xi, . . . , x;), where 
yj G S^J"^ for j > I, with the following properties, yp = yg for each p, q £ ai,i > 1. 
Let T{yi+i, . . . ,yfc) := T x (g)^^;_j_-^yj. Then T{yi+i, . . . ,yfe) G Sym(ni,/). Theorem 
M yields that T{yi-\-i, ■ ■ ■ ,yk) best rank one approximation of the form c ®j=i yj 
where yi = . . . = y^. Hence the rank one approximation c 0j=i yj is symmetric 
with respect ai, . . . , a^. □ 



6 Uniqueness of symmetric rank one approximation for 
generic symmetric tensors 

For X = (xi, . . .,Xny G C" denote ||x|| = '\/J27=i l^iP- % ^'^C we denote the 
tensor products of d copies of C''. Then ^'^'x G (g)^C" is a decomposable tensor 
X (8) ... (g) X. Denote by Sym(n, d, C) C iJSj'^C" the space of all symmetric tensors 
T = ['l^ii,--;id\h= =id=i ^i^^ complex entries. It is well known that Sym(n,d) is 
isomorphic to c( 



/n + d — l\ 



Theorem 13 Let d > 3,n > 2 be integers. Then there exists an algebraic 
variety T,{n,d) C Sym(n, d, C) such that for each T G Sym((i, n, C) \ T,{n,d) the 
symmetric eigensystem 

rx®'^~^x = x, x/0 (6.1) 

have at most {d — 1)" — 1 distinct solutions, which are invariant under the multipli- 
cation by d — 2 roots of unity. 

Assume furthermore that T G Sym(d, n) \ T,{n,d). If d > 3 is odd, the number 
of real distinct solutions of the above system is at most - — — . Furthermore any 
solution of 

T X (gj'^-ix = -X, X / (6.2) 



is the negative of the real solution of (j6.ip . 

// d is even then the systems (|6.ip - (j6.2p all together have at most ^^^'^^j_2 
real solutions which are invariant by multiplication by —1. 



Proof. Let T G Sym(n,d, C). Consider the system 

r X ^'^-^x = 0, X G C". (6.3) 

Let ui,...,u„ G C" be linearly independent. Let To = J27=i^'^'^i- Then the 
above system is equivalent to Yli=ii^J'^Y^^^i = 0. Since ui, . . . ,u„ are linearly 



10 



independent it follows that (u^x)*^ ^ = for i E [n]. Hence (u^x) = for i G [n]. 
As ui, . . . , u„, independent it follows that x = 0. That is 

mm \\To X ®^"^x|| > 0. 

||x|| = l,xgC" 

The above inequality holds for T S Sym(n, d, C) in some neighborhood of To- Hence 
there exists a strict algebraic variety S(n, d) C Sym(n,(i, C) such that for each 
T G Sym(n, d, C) \ S(n, d) the system (j6.3p has a unique solution x = 0. 
Assume that T G Sym(n, d, C) \ S(n, d). Consider the polynomial system 

T X ^j-^-^x - X = 0, X G C". (6.4) 

Its principal part is the homogeneous system (j6.3p . where each homogeneous poly- 
nomial is of degree d—1. Hence (j6.4p has at most {d— 1)"" distinct solutions. (This 
is the precise version of Bezout's theorem. See for example [2].) Note that x = is 
a solution. Hence ()6.ip has at most [d — 1)" — 1 distinct solutions. Clearly, if x 7^ 
is a solution of (|6.ip then Q'x. is also a solution of (16. ip if (^^^"2 _ As x 7^ 0, note 
that Cx 7^ ?7x if ^, r/ are two distinct d — 2 roots of 1. That establishes the first part 
of the of the theorem. 

Assume now that T G Sym(n, d) \ S((i, n). We know that (16. ip has at most 
(d — 1)" — 1 complex distinct solutions. Assume that x 7^ is a real solution. Then 
Q'x. is also a solution for C,'^~'^ = 1. Suppose that d > 3 is odd. Then ("x is real when 
C^^^ = 1 if and only if C = 1- Hence each real solution of (|6.ip gives rise to another 
d — 3 distinct nonreal solutions of (16. ip . Hence the number of real solutions of (16. ip 
is at most — — . Note that for any real solution x of (16. ip — x is a real solution 
of ()6.2p and vice versa. This proves our theorem for d > 3 odd. 

Assume now that d > 3 is even. Then for any real solution x of (|6.ip or (|6.2p 
we get another real solution — x, and d — 4 complex solutions of (16.11) or (16. 2p . 
respectively, of the form ("x, where ^"^"^ = l^^ ^ ±1. Assume now that x is a real 
solution (j6.2p . Then t^x, r/*^"^ = — 1 gives rise to d — 2 distinct nonreal solutions 
of (j6.ip . Hence the total number of real solutions of the systems (j6.ip ~ (j6.2p is 
2((rf-i)"-i) n 

d-2 ■ 

Theorem 14 There exists a subvariety Si C Sym(n,d, C), where Si(n,d) 5 
S(n,d), wzi/i the following properties. Let T G Sym(n, d) \ Si(d, n) and consider 
nonzero critical values {T, (SD'^x) on S"~^ and t/ie corresponding critical points. Then 

1. For an odd d > 3 there are N{T) < ^^^^^^2 — nonzero distinct critical values, 
where each critical value X has a unique corresponding critical point x G S*^"^. 
Furthermore, —X is also a critical value with the corresponding unique critical 
point —X. 

2. For an even d > 3 there are N(T) < '^^^'^ d-2 nonzero distinct critical 
values, where each critical value X has exactly two critical points itx G S"~^. 
Furthermore, the absolute values of two distinct critical points are distinct. 

Proof. Let T G Sym(n,d, C) \ S(n, d). We claim that there exists a subvariety 
So(d, n) C Sym(n,d, C) such that for each T So(n,d) the following conditions 
hold. 
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1. Assume that d > 3 is odd. Then the system (j6.ip has {d — 1)" — 1 solutions, 
and for two different solutions x, y the inequality x^x ^ y^y holds. 

2. Assume that d > 3 is even. Then the systems (|6.ip and (j6.2p each has {d — 
!)"■ — 1 solutions, and for two different solutions x, y of either (j6.ip or (j6.2p . 
where y — x, the inequality x~''x 7^ y'''y holds. 

We consider a special tensor T = Y17=i *i ^^'S^'^ej, where = {Su, . . . , Sni)~^ ,ti € 
C \ {0} for i G [n]. For this tensor we can explicitly calculate all solutions of 
(|6.ip . Namely, it is of the form (xi, . . . ,Xn)'^ where each satisfies the equation 
_ — Q_ (•"\Ye need to exclude from this list the trivial solution x = 0.) This 
means that (j6.ip has {d — 1)" — 1 solutions. Moreover, the solutions of (|6.2p are of 
the form 0x, where x is a solution of (j6.ip and is a fixed solution of 9'^~'^ = — 1. 
To be explicit, let m > be the integer such that is an odd integer. Then 

:= e^^. 

Let F = Q[^] be a finite extension field of the rational numbers Q, where ^ is 
a primitive root of ^^^^^ = 1. (Each element of F is a polynomial of degree d — 3 
at most with rational coefficients.) Assume that tf, . . . are linearly independent 
over Q[^]. 

We first consider the simple case: d > 3 is odd. We claim that if x = {xi, . . . , 
and y = (2/1, . . . ,yn)~^ are different nonzero solution of (j6.ip then x^x 7^ y^y. In- 
deed X = (Ci^i, • • • ,Cn^n)^;y = (f?!^!, • • • jVntn)'^ ■ The assumption that x and y 
satisfy (|6.ip imply that each Q and r/j satisfy the equation s{s'^~'^ — 1) = 0. Observe 
next that 

n 

xTx-yV = E(C'-^'K'- (6-5) 

i=l 

Suppose that x'''x — y'''y = 0. As tf , . . . , are linearly independent over F it 
follows that (f = Tjf for i £ [d]. So Q = zizrji for i G [d]. Clearly d = <=^ 7]i = 0. 
Assume that (i + 0. So Cf"^ = 1 ^ r/f"^ = 1. Hence Ci = m for i G [d]. So x = y 
contrary to our assumption. Hence, there exists a subvariety So(n, d) C Sym(n, d, C) 
such that for T G Sym(n, d, C) \ d) U 5]o("-) d)) the condition [1] hold. 

Assume now that d > 3 is even. Let x = (Ci^i, • • • , Cn^n)"'' and y = {r\\t\^ . . . , rj^tn)^ 
satisfy either ([61]) or So Ci(('/'0)''"^) = ^[k^mY''^ - 1) = for i G [d]. Here 

0, V' £ {1)^}- Assume that x^x — yy^ = 0. So Ci = i^i for i G [d]. Hence cf) = 9. 
That is either x and y satisfy (|6.ip or x and y satisfy (j6.2p . However, it is possible 
that x'''x — y~'^y = and x 7^ ity. We now find a tensor T' G Sym(n, d, C) in a 
small neighborhood of T such that for x',y', where x' 7^ ±y', satisfying either the 
system (16. ip or (16. 2p for T', one has the inequality (x')^x' — (y')^y' 7^ 0. Since any 
solution X of ()6.ip or ()6.2p is a simple solution, (as we have the maximal number 
of distinct solutions), we can use an implicit function theorem to find the unique 
solutions x' of (j6.ip or (|6.2p in the neighborhood of the solution x respectively. 
Since for x'''x — y^y = may hold if only x, y satisfy both either (16. ip or (16. 2p it 
is enough to show that (x')'''x' — (y')^y' 7^ where x',y' satisfy (16. ip . 

Let T(e) ■.= T + eS for a fixed S G Sym(n, d, C). Consider the system 

r(£) X ®'^-ix(e) =x(e). (6.6) 
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The implicit function theorem yields that x(e) can be expanded in power series of 
e. So 

x(e) = x + exi + 0(e2). (6.7) 

Denote by 7~(x) := T x (gi'^^^x G Sym(n,2,C). The system (j6.ip is equivalent to 
T(x)x = X and x^T(x) = x^. Then the e term in (I6.6p is 

(d- l)r(x)xi +5 X ^^^-^x = xi. (6.8) 

Multiply the above equality by x^ to deduce that 



Observe finally that 



x(e)'^x(e) - y(e)^y(e) = x^x - y^y + ^(5, ^'^x - ^^'^y) + 0{e^). (6.10) 

Note that (8)'^x — (X''^y is zero tensor if and only if x = 7y where 7*^ = 1. 
Hence we can choose 5 G Sym(ra, d, C) lying outside a finite number of subspaces of 
codimension one, such that the coefficient of e is different from zero for each pair 
of solutions x,y of (|6.ip such that x ^ ity and x^x = y^y. Hence, there exists 
a subvariety So(n, d) C Sym(n, d, C) such that for T G Sym(n, d, C) \ (S(n,(i) U 
Eo(n, d)) the condition [2] hold. 

Let Si(n,(i) = S(n,(i) U So(n, d) and T G Sym(n, d) \ Si(n,(i). Assume that 
A 7^ is a critical value of (T, (8''^z) on S"~^ with a corresponding critical point 
y G S"-^ Hence T x ®'^-'^y = Ay. 

Assume first that d > 3 is odd. Clearly, — y is a critical point corresponding 
to the critical value —A. Without loss of generality we may assume that A > 0. 
Then x := X~"^y is a real solution of (|6.ip . Vice versa, any real solution of x of 
()6.ip gives rise to a critical point y = with the critical value ||x||~('^~^). Recall 
that for T G Sym(n,(i) \ Si(n,(i) (|6.ip has exactly {d — 1)" — 1 different complex 
solutions, with corresponding (d — 1)" — 1 different values of x^x. Hence (j6.ip has 
at most — — real solutions and the length of all these solutions are distinct. 
Therefore, the number of positive nonzero critical points of (T, CS'^y) on S"~^ is 
at most ^'^ J}_2 — and to each critical positive value corresponds a unique critical 
point. These arguments prove the theorem for d odd. 

Assume now that d is even. The — y is a critical point of the critical value 
A. Let x+ := A^^^y if A > and x_ := {-Xy^y if A < 0. Then x+ sat- 
isfies (j6.ip and x_ satisfies (j6.2p . Theorem 1131 vields that the number of critical 
nonzero values of {T, ®'^'z,) is at most ^^^'^"^^2 ■ is left to show that the absolute 
values of nonzero critical values are distinct. Let x, y be two solutions of either 
()6.ip or ()6.2|) . and assume that x 7^ iby. Then u := n^x, v := jj^y are critical 

points of (T, (X'^'z) on S"^^, corresponding to critical values A, fi respectively. Clearly 
|A| = ||x||~('^-2),|A| = llyir^'^"^). Since T Si(n,d) we deduce that x^x / y^y. 
So|A|/|^|. □ 



Corollary 15 Let d > 3 be an integer and assume that T G Sym(n, d)\Si(n, d). 
Then T has unique symmetric best rank one approximation. 
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Proof. Best rank one approximation corresponds to the critical point y of 
(T, (gi'^z), corresponding to the critical value A where is maximal. The corre- 
sponding best rank one symmetric approximation is A := (T, <SD'^y) (E>'^ y- So — y 
gives the same A. For d odd all positive critical values are distinct. Hence A is 
unique. For d even, each nonzero critical value has two critical points ity. The 
absolute values of the critical values are distinct. Hence A is unique. □ 
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